Skip to content

Hide native Windows 11 Start button when using a replacement - #2547

Open
YellowNest wants to merge 34 commits into
Open-Shell:masterfrom
YellowNest:fix/win11-native-start-button-overlap
Open

YellowNest wants to merge 34 commits into
Open-Shell:masterfrom
YellowNest:fix/win11-native-start-button-overlap

Conversation

@YellowNest

@YellowNest YellowNest commented Oct 3, 2026 •

Copy link
Copy Markdown

Summary

Hide the native Windows 11 Start button visuals while Open-Shell's replacement Start button is enabled.

Windows 11 renders the Start button through XAML, so the older HWND-based hiding logic cannot remove the native glyph. This caused custom Open-Shell buttons to be drawn on top of the Windows Start icon.

The change uses the Windows XAML diagnostics visual tree to:

  • collapse the native Start glyph
  • disable hit testing on the underlying native Start control
  • preserve the native control's layout slot
  • restore the original XAML properties when replacement is disabled or Open-Shell exits
  • respect the All taskbars setting

The replacement button itself is not moved or resized by this code, so Windows remains responsible for centered taskbar layout.

Verification

Tested on Windows 11 25H2 with centered taskbar:

  • Custom replacement button no longer shows the Windows Start glyph underneath.
  • Replacement button remains in the correct centered Start position.
  • Left-click and right-click behavior continue to work.
  • Disabling Replace Start button restores the native Windows Start button.
  • Re-enabling it hides the native button again.
  • Exiting Open-Shell restores the native Start button without restarting Explorer.

GitHub Actions Build completed successfully for the current head e9039e3:

https://github.com/Open-Shell/Open-Shell-Menu/actions/runs/37805091803

The lifecycle implementation avoids loading StartMenuHelper and initializing XAML Diagnostics when the replacement Start button is disabled.

Repeated Explorer exit/restart and complete injected DLL unloading still require runtime verification. A successful build does not establish those behaviors.

Fixes #1631

@ge0rdi

ge0rdi commented Oct 4, 2026

Copy link
Copy Markdown
Member

This is super awesome.
I knew about the Explorer TAP interface from TranslucentTB project. But never managed to properly use it inside Open-Shell.

Glad to see it works nicely. And I guess it should be used for more customizations and improvements related to start button and taskbar itself.
For example there is this very ugly mouse hook HookDesktopThreadMouse that we are using to intercept mouse events that would be consumed by XAML framework otherwise. It would be great to get rid of this by changing XAML handler for start button.

Do you plan to work more in this area? I think it would be really appreciated.

@ge0rdi

ge0rdi commented Oct 4, 2026

Copy link
Copy Markdown
Member

I have noticed that enabling custom Start button cause also that TaskView button dissapears. There is just empty place instead of the button.
image

@ge0rdi

ge0rdi commented Oct 4, 2026

Copy link
Copy Markdown
Member

Another issue is that when I Exit Open-Shell (via right-click menu), it is not possible to start it again.

Starting Open-Shell Menu Settings shortcut doesn't do anything (no settings are shown, Start menu is not customized).

@YellowNest

Copy link
Copy Markdown
Author

Thanks for testing this carefully. Both reports were valid and pointed to separate problems in the initial TAP implementation.

I pushed the follow-up in 9f572bb and ran a fresh Actions build against that exact commit; the build passes.

The Task View regression came from treating every ExperienceToggleButton#LaunchListButton as the Start control. Current Windows 11 uses that shape for multiple taskbar controls. The code now resolves the attached AutomationId and only classifies the actual StartButton for suppression, so Task View is left alone.

The restart problem was a separate TAP lifetime issue. Shutdown now restores the XAML properties synchronously, removes the visual-tree callback with UnadviseVisualTreeChange, tears down the dispatch window, and balances the additional module reference introduced by InitializeXamlDiagnosticsEx. There is also a shutdown guard preventing a late connection attempt from attaching again while Open-Shell is exiting.

The build for the follow-up is here:
https://github.com/YellowNest/Open-Shell-Menu/actions/runs/37219583485

The two paths that still deserve runtime confirmation are:

  • custom Start button enabled while Task View remains visible and functional
  • Exit Open-Shell, then start Open-Shell again without restarting Explorer

And yes, I plan to keep working in this area. Replacing HookDesktopThreadMouse with a narrower XAML-side solution looks like a good next target. I want to keep this PR focused and make sure the TAP lifecycle is solid first, then tackle that separately.

@ge0rdi

ge0rdi commented Oct 4, 2026

Copy link
Copy Markdown
Member

I want to keep this PR focused and make sure the TAP lifecycle is solid first, then tackle that separately.

Definitely.
Each fix/feature deserves own PR.

I'm glad you plan to work on this more.

@YellowNest

Copy link
Copy Markdown
Author

Thanks, I really appreciate that.

I use Open-Shell myself, so contributing here is useful to me too. I care about keeping it working well on current Windows builds, and if I can help take some of the load off, I'm happy to.

I also appreciate the trust you're putting in the work. Your reviews and testing are valuable, especially around these Windows 11 edge cases, so I'll keep changes focused and separate like you suggested.

I've already started looking at the XAML input side in a separate research branch. The first build is clean, but I'll keep it out of upstream until I've tested the behavior properly.

Glad to help.

@ge0rdi

ge0rdi commented Oct 4, 2026

Copy link
Copy Markdown
Member

custom Start button enabled while Task View remains visible and functional

I can confirm this works properly now.

Exit Open-Shell, then start Open-Shell again without restarting Explorer

I'm now experiencing random Explorer crashes when Exiting Open-Shell and starting it again (using Open-Shell Menu Settings shortcut). This is on 26H2 (Windows Sandbox).
I was able to obtain one dump and the stack looked like:

  *** Stack trace for last set context - .thread/.cxr resets it
 # Child-SP          RetAddr               Call Site
00 00000000`0367d600 00007ffc`8e616071     combase!Ordinal130+0x358
01 00000000`0367d650 00007ffc`8e62eec4     combase!Ordinal95+0x2e11
02 00000000`0367d730 00007ffc`8e62eae1     combase!Ordinal164+0x35f4
03 00000000`0367d800 00007ffc`8e558579     combase!Ordinal164+0x3211
04 00000000`0367d850 00007ffc`8e5a6543     combase!NdrOleDllGetClassObject+0x4c99
05 00000000`0367d890 00007ffc`8e5a63f6     combase!CoLockObjectExternal+0x463
06 00000000`0367d8c0 00007ffc`8e65f463     combase!CoLockObjectExternal+0x316
07 00000000`0367d8f0 00007ffc`8e62e705     combase!Ordinal130+0x1993
08 00000000`0367d960 00007ffc`8e4e6072     combase!Ordinal164+0x2e35
09 00000000`0367da20 00007ffc`6eb329f0     combase!CoReleaseMarshalData+0x122
0a 00000000`0367daf0 00007ffc`6eb31f64     oleacc!ReleaseMarshallData+0xac
0b 00000000`0367db30 00007ffc`6eb31ef2     oleacc!AccInfo::PropInfo::~PropInfo+0x54
0c 00000000`0367db60 00007ffc`6eb320c5     oleacc!AccInfo::PropInfo::`scalar deleting destructor'+0xe
0d 00000000`0367db90 00007ffc`6eb3202b     oleacc!AccInfo::~AccInfo+0x55
0e 00000000`0367dbc0 00007ffc`6eb31091     oleacc!CPropMgrImpl::RemoveEntry+0x4b
0f 00000000`0367dbf0 00007ffc`6eb30cbe     oleacc!CPropMgrImpl::Clean+0xed
10 00000000`0367dc40 00007ffc`6eb30bc4     oleacc!CPropMgrImpl::SetPropValue+0x2a
11 00000000`0367dcc0 00007ffc`5a861615     oleacc!CPropMgr::SetHwndPropStr+0xe4
12 00000000`0367dd90 00007ffc`5a847687     StartMenuDLL!SetControlAccessibleName+0x75 [D:\a\Open-Shell-Menu\Open-Shell-Menu\Src\Lib\SettingsUIHelper.cpp @ 166] 
13 00000000`0367dde0 00007ffc`5a846769     StartMenuDLL!CSettingsDlg::OnInitDialog+0x727 [D:\a\Open-Shell-Menu\Open-Shell-Menu\Src\Lib\Settings.cpp @ 1316] 
14 00000000`0367df70 00007ffc`5a82ee28     StartMenuDLL!CSettingsDlg::ProcessWindowMessage+0x69 [D:\a\Open-Shell-Menu\Open-Shell-Menu\Src\Lib\Settings.cpp @ 1116] 
15 00000000`0367e010 00007ffc`6df61148     StartMenuDLL!ATL::CDialogImplBaseT<ATL::CWindow>::DialogProc+0x78 [C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\atlmfc\include\atlwin.h @ 3932] 
16 00000000`0367e0b0 00007ffc`8eeb426e     atlthunk!AtlThunk_0x06+0x18
17 00000000`0367e0f0 00007ffc`8eeb3604     user32!UserCallDlgProcCheckWow+0x1f2
18 00000000`0367e1e0 00007ffc`8eeb3526     user32!DefDlgProcWorker+0xc4
19 00000000`0367e290 00007ffc`8eeb52e6     user32!DefDlgProcW+0x36
1a 00000000`0367e2d0 00007ffc`8eeb4b6c     user32!UserCallWinProcCheckWow+0x356
1b 00000000`0367e430 00007ffc`8eedf593     user32!DispatchClientMessage+0x9c
1c 00000000`0367e490 00007ffc`906c5c04     user32!_fnDWORD+0x33
1d 00000000`0367e4f0 00007ffc`8e251384     ntdll!KiUserCallbackDispatcherContinue
1e 00000000`0367e578 00007ffc`8eeb73c6     win32u!NtUserMessageCall+0x14
1f 00000000`0367e580 00007ffc`8eec0192     user32!SendMessageWorker+0xa26
20 00000000`0367e620 00007ffc`8eebefc7     user32!InternalCreateDialog+0x1032
21 00000000`0367e800 00007ffc`8eec0318     user32!CreateDialogIndirectParamAorW+0x57
22 00000000`0367e850 00007ffc`5a84a2f6     user32!CreateDialogIndirectParamW+0x18
23 (Inline Function) --------`--------     StartMenuDLL!CResizeableDlg<CSettingsDlg>::Create+0xa6 [D:\a\Open-Shell-Menu\Open-Shell-Menu\Src\Lib\SettingsUIHelper.h @ 55] 
24 00000000`0367e890 00007ffc`5a82d00d     StartMenuDLL!EditSettings+0x1e6 [D:\a\Open-Shell-Menu\Open-Shell-Menu\Src\Lib\Settings.cpp @ 1934] 
25 00000000`0367e920 00007ffc`5a792e1d     StartMenuDLL!EditSettings+0x32d [D:\a\Open-Shell-Menu\Open-Shell-Menu\Src\StartMenu\StartMenuDLL\SettingsUI.cpp @ 5287] 
26 00000000`0367ea50 00007ffc`8eed3bed     StartMenuDLL!HookDesktopThread+0x48d [D:\a\Open-Shell-Menu\Open-Shell-Menu\Src\StartMenu\StartMenuDLL\StartMenuDLL.cpp @ 3694] 
27 00000000`0367f2a0 00007ffc`8eebb838     user32!DispatchHookW+0xad
28 00000000`0367f330 00007ffc`8eebb77c     user32!CallHookWithSEH+0x28
29 00000000`0367f380 00007ffc`906c5c04     user32!_fnHkINLPMSG+0x7c
2a 00000000`0367f3d0 00007ffc`8e251304     ntdll!KiUserCallbackDispatcherContinue
2b 00000000`0367f488 00007ffc`8eeb19df     win32u!NtUserPeekMessage+0x14
2c 00000000`0367f490 00007ffc`8eeb1968     user32!_PeekMessage+0x3f
2d 00000000`0367f500 00007ff7`4081a271     user32!PeekMessageW+0x168
2e 00000000`0367f570 00007ff7`40818aeb     explorer!CTray::_MessageLoop+0x2c1
2f 00000000`0367f6b0 00007ffc`8fab3a8c     explorer!CTray::MainThreadProc+0x7b
30 00000000`0367f710 00007ffc`8e8bcdf7     SHCore!_WrapperThreadProc+0x15c
31 00000000`0367f800 00007ffc`905dee4c     kernel32!BaseThreadInitThunk+0x17
32 00000000`0367f830 00000000`00000000     ntdll!RtlUserThreadStart+0x2c

I can share the dump eventually.

@YellowNest

Copy link
Copy Markdown
Author

Thanks for the dump trace. I treated this as a teardown/lifetime problem rather than an accessibility-code failure and reworked the TAP shutdown path.

The latest head is 192b2e3.

The important changes are:

  • keep the module reference acquired by InitializeXamlDiagnosticsEx until after the visual-tree callback and XAML/COM references are fully released
  • run the XAML/COM teardown on the same thread that owns the TAP dispatch window
  • only publish g_Tap after AdviseVisualTreeChange succeeds, with complete cleanup on failed setup paths
  • hold a separate StartMenuDLL module reference for the asynchronous connection worker, released atomically with FreeLibraryAndExitThread, so Exit cannot unload the DLL while that worker is still executing

I also checked the current upstream master changes; they don't overlap this code.

The full upstream PR build for 192b2e3 is green:
https://github.com/Open-Shell/Open-Shell-Menu/actions/runs/37225846278

The critical runtime test now is repeated cycles on 26H2:

  1. enable the custom Start button
  2. Exit Open-Shell
  3. start it again through Open-Shell Menu Settings
  4. repeat several times without restarting Explorer

If Explorer still crashes, the full dump would be useful. I don't want to paper over this with another timing workaround; the remaining failure, if any, should be traced from the dump.

Comment thread Src/StartMenu/StartMenuHelper/Win11StartButtonTap.cpp Outdated
Comment thread Src/StartMenu/StartMenuHelper/Win11StartButtonTap.cpp Outdated
Comment thread Src/StartMenu/StartMenuHelper/Win11StartButtonTap.cpp Outdated
Comment thread Src/StartMenu/StartMenuHelper/Win11StartButtonTap.cpp Outdated
Comment thread Src/StartMenu/StartMenuHelper/Win11StartButtonTap.cpp Outdated
@ge0rdi
ge0rdi self-requested a review October 6, 2026 19:42

HRESULT Deactivate( void )
{
// The diagnostics runtime retains this site beyond Open-Shell's lifetime.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may need to call m_Visual->UnadviseVisualTreeChange here.

It seems that when I exit Open-Shell, StartMenuHelper64.dll stays loaded in explorer. Something is still holding reference to it. I think it is the XAML diagnostic framework.

CWin11StartButtonTap destructor is never called (I have breakpoint there).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in bee430c. Exit now restores our XAML overrides synchronously before unregistering the callback, then calls UnadviseVisualTreeChange, clears the replay state, drains queued apply work and tears down the dispatch window.

One distinction is intentional: StartMenuHelper64.dll may still remain mapped after a successful InitializeXamlDiagnosticsEx. The public XAML diagnostics surface has no session-uninitialize API; UnadviseVisualTreeChange only unregisters visual-tree mutation callbacks. Forcing the injected module out while the diagnostics runtime still owns the TAP object would be unsafe. Open-Shell-owned subscription/window/state is now fully detached on Exit, while Start can re-advise the retained diagnostics service without creating another diagnostics session.

The runtime path worth retesting is repeated Exit -> Open-Shell Menu Settings restart on 26H2.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The public XAML diagnostics surface has no session-uninitialize API; UnadviseVisualTreeChange only unregisters visual-tree mutation callbacks.

It seems that after UnadviseVisualTreeChange the XAML framework frees TAP object because CWin11StartButtonTap destructor is called.

Yet, StartMenuHelper64.dll stays loaded. I'm not sure what is holding it. But normally (in actual Open-Shell) it is unloaded after some time (like few minutes or so).

I was also trying to use UWPSpy that also uses TAP mechanism. It has DLL that is loaded into examined process and when you close the tool, the DLL gets unloaded.

I mean this is probably no big deal, because normally one doesn't exit Open-Shell.
It may just cause issues during update, where we will now need to restart Explorer for sure (previously it was not always necessary).

Comment thread Src/StartMenu/StartMenuHelper/Win11StartButtonTap.cpp
Comment thread Src/StartMenu/StartMenuHelper/Win11StartButtonTap.cpp Outdated
Comment thread Src/StartMenu/StartMenuHelper/Win11StartButtonTap.cpp Outdated
@ge0rdi

ge0rdi commented Oct 7, 2026

Copy link
Copy Markdown
Member

The crash that I mentioned in #2547 (comment) is actually not related to changes in this PR.

I can replicate it with official 4.4.201 build.
I have created #2556 ticket for that.

So far have no clue what could be wrong.
I'd appreciate any help on that.

@ge0rdi ge0rdi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry. but I have feeling like it gets more complicated with each batch of changes :(
I'd rather keep it simpler (as the idea is rather simple).

I'll try to provide some more comments, just don't have energy for it now.

~CWin11StartButtonTap( void )
{
{
std::unique_lock lock(g_TapMutex);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't feel right. We should rather remove g_Tap reference when we are deactivating TAP (Deactivate method or StopWin11StartButtonTap perhaps).

}

{
std::lock_guard lock(m_LifecycleMutex);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we are in object destructor. That means no other threads are supposed to use this object (otherwise the behavior would be undefined).
So there is no need to hold object mutexes here.
Moreover it makes no sense to call Unadvise here - we had to call it before, otherwise we won't get here.
Also no point to ResetElements - nobody can use those anymore.
We should just destroy dispatch window and that's it.

// Never leave a window pointing at an object that is being destroyed.
HWND dispatch = m_Dispatch;
if (dispatch)
SetWindowLongPtr(dispatch, GWLP_USERDATA, 0);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this belongs to DestroyDispatchWindow. That function is responsible for destroying the window so one of first things it should do is to reset user data.

Here in destructor we should just call DestroyDispatchWindow and that's it.

return false;
}

HRESULT RequestApply( bool synchronous, bool enabled )

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This functions looks overly complicated.
It is supposed to just pass some message.
Why to complicate it with all those error handlings and timeouts.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Custom start button overlayed on top of original Windows 11 one

2 participants